{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Introduction to DataFrames\n",
    "**[Bogumił Kamiński](http://bogumilkaminski.pl/about/), May 23, 2018**\n",
    "\n",
    "Let's get started by loading the `DataFrames` package."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "using DataFrames"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Constructors and conversion"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Constructors\n",
    "\n",
    "In this section, you'll see many ways to create a `DataFrame` using the `DataFrame()` constructor.\n",
    "\n",
    "First, we could create an empty DataFrame,"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th></tr></thead><tbody></tbody></table>"
      ],
      "text/plain": [
       "0×0 DataFrames.DataFrame\n"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "DataFrame() # empty DataFrame"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Or we could call the constructor using keyword arguments to add columns to the `DataFrame`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>A</th><th>B</th><th>C</th></tr></thead><tbody><tr><th>1</th><td>1</td><td>0.865057</td><td>Jds</td></tr><tr><th>2</th><td>2</td><td>0.870442</td><td>hDG</td></tr><tr><th>3</th><td>3</td><td>0.128666</td><td>oyt</td></tr></tbody></table>"
      ],
      "text/plain": [
       "3×3 DataFrames.DataFrame\n",
       "│ Row │ A │ B        │ C   │\n",
       "├─────┼───┼──────────┼─────┤\n",
       "│ 1   │ 1 │ 0.865057 │ Jds │\n",
       "│ 2   │ 2 │ 0.870442 │ hDG │\n",
       "│ 3   │ 3 │ 0.128666 │ oyt │"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "DataFrame(A=1:3, B=rand(3), C=randstring.([3,3,3]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can create a `DataFrame` from a dictionary, in which case keys from the dictionary will be sorted to create the `DataFrame` columns."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>A</th><th>B</th><th>C</th></tr></thead><tbody><tr><th>1</th><td>1</td><td>true</td><td>'a'</td></tr><tr><th>2</th><td>2</td><td>false</td><td>'b'</td></tr></tbody></table>"
      ],
      "text/plain": [
       "2×3 DataFrames.DataFrame\n",
       "│ Row │ A │ B     │ C   │\n",
       "├─────┼───┼───────┼─────┤\n",
       "│ 1   │ 1 │ true  │ 'a' │\n",
       "│ 2   │ 2 │ false │ 'b' │"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = Dict(\"A\" => [1,2], \"B\" => [true, false], \"C\" => ['a', 'b'])\n",
    "DataFrame(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Rather than explicitly creating a dictionary first, as above, we could pass `DataFrame` arguments with the syntax of dictionary key-value pairs. \n",
    "\n",
    "Note that in this case, we use symbols to denote the column names and arguments are not sorted. For example, `:A`, the symbol, produces `A`, the name of the first column here:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>A</th><th>B</th><th>C</th></tr></thead><tbody><tr><th>1</th><td>1</td><td>true</td><td>'a'</td></tr><tr><th>2</th><td>2</td><td>false</td><td>'b'</td></tr></tbody></table>"
      ],
      "text/plain": [
       "2×3 DataFrames.DataFrame\n",
       "│ Row │ A │ B     │ C   │\n",
       "├─────┼───┼───────┼─────┤\n",
       "│ 1   │ 1 │ true  │ 'a' │\n",
       "│ 2   │ 2 │ false │ 'b' │"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "DataFrame(:A => [1,2], :B => [true, false], :C => ['a', 'b'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here we create a `DataFrame` from a vector of vectors, and each vector becomes a column."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>x1</th><th>x2</th><th>x3</th></tr></thead><tbody><tr><th>1</th><td>0.738969</td><td>0.476396</td><td>0.926968</td></tr><tr><th>2</th><td>0.498559</td><td>0.190063</td><td>0.839678</td></tr><tr><th>3</th><td>0.0957712</td><td>0.843156</td><td>0.120698</td></tr></tbody></table>"
      ],
      "text/plain": [
       "3×3 DataFrames.DataFrame\n",
       "│ Row │ x1        │ x2       │ x3       │\n",
       "├─────┼───────────┼──────────┼──────────┤\n",
       "│ 1   │ 0.738969  │ 0.476396 │ 0.926968 │\n",
       "│ 2   │ 0.498559  │ 0.190063 │ 0.839678 │\n",
       "│ 3   │ 0.0957712 │ 0.843156 │ 0.120698 │"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "DataFrame([rand(3) for i in 1:3])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " For now we can construct a single `DataFrame` from a `Vector` of atoms, creating a `DataFrame` with a single row. In future releases of DataFrames.jl, this will throw an error."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[1m\u001b[33mWARNING: \u001b[39m\u001b[22m\u001b[33mpassing columns argument with non-AbstractVector entries is deprecated\u001b[39m\n",
      "Stacktrace:\n",
      " [1] \u001b[1mdepwarn\u001b[22m\u001b[22m\u001b[1m(\u001b[22m\u001b[22m::String, ::Symbol\u001b[1m)\u001b[22m\u001b[22m at \u001b[1m.\\deprecated.jl:70\u001b[22m\u001b[22m\n",
      " [2] \u001b[1m#DataFrame#57\u001b[22m\u001b[22m\u001b[1m(\u001b[22m\u001b[22m::Bool, ::Type{T} where T, ::Array{Float64,1}, ::Array{Symbol,1}\u001b[1m)\u001b[22m\u001b[22m at \u001b[1mD:\\Software\\JULIA_PKG\\v0.6\\DataFrames\\src\\dataframe\\dataframe.jl:154\u001b[22m\u001b[22m\n",
      " [3] \u001b[1mDataFrames.DataFrame\u001b[22m\u001b[22m\u001b[1m(\u001b[22m\u001b[22m::Array{Float64,1}\u001b[1m)\u001b[22m\u001b[22m at \u001b[1mD:\\Software\\JULIA_PKG\\v0.6\\DataFrames\\src\\dataframe\\dataframe.jl:152\u001b[22m\u001b[22m\n",
      " [4] \u001b[1minclude_string\u001b[22m\u001b[22m\u001b[1m(\u001b[22m\u001b[22m::String, ::String\u001b[1m)\u001b[22m\u001b[22m at \u001b[1m.\\loading.jl:522\u001b[22m\u001b[22m\n",
      " [5] \u001b[1mexecute_request\u001b[22m\u001b[22m\u001b[1m(\u001b[22m\u001b[22m::ZMQ.Socket, ::IJulia.Msg\u001b[1m)\u001b[22m\u001b[22m at \u001b[1mD:\\Software\\JULIA_PKG\\v0.6\\IJulia\\src\\execute_request.jl:158\u001b[22m\u001b[22m\n",
      " [6] \u001b[1m(::Compat.#inner#17{Array{Any,1},IJulia.#execute_request,Tuple{ZMQ.Socket,IJulia.Msg}})\u001b[22m\u001b[22m\u001b[1m(\u001b[22m\u001b[22m\u001b[1m)\u001b[22m\u001b[22m at \u001b[1mD:\\Software\\JULIA_PKG\\v0.6\\Compat\\src\\Compat.jl:385\u001b[22m\u001b[22m\n",
      " [7] \u001b[1meventloop\u001b[22m\u001b[22m\u001b[1m(\u001b[22m\u001b[22m::ZMQ.Socket\u001b[1m)\u001b[22m\u001b[22m at \u001b[1mD:\\Software\\JULIA_PKG\\v0.6\\IJulia\\src\\eventloop.jl:8\u001b[22m\u001b[22m\n",
      " [8] \u001b[1m(::IJulia.##14#17)\u001b[22m\u001b[22m\u001b[1m(\u001b[22m\u001b[22m\u001b[1m)\u001b[22m\u001b[22m at \u001b[1m.\\task.jl:335\u001b[22m\u001b[22m\n",
      "while loading In[7], in expression starting on line 1\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>x1</th><th>x2</th><th>x3</th></tr></thead><tbody><tr><th>1</th><td>0.613673</td><td>0.9388</td><td>0.976714</td></tr></tbody></table>"
      ],
      "text/plain": [
       "1×3 DataFrames.DataFrame\n",
       "│ Row │ x1       │ x2     │ x3       │\n",
       "├─────┼──────────┼────────┼──────────┤\n",
       "│ 1   │ 0.613673 │ 0.9388 │ 0.976714 │"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "DataFrame(rand(3))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Instead use a transposed vector if you have a vector of atoms (in this way you effectively pass a two dimensional array to the constructor which is supported)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>x1</th><th>x2</th><th>x3</th></tr></thead><tbody><tr><th>1</th><td>1</td><td>2</td><td>3</td></tr></tbody></table>"
      ],
      "text/plain": [
       "1×3 DataFrames.DataFrame\n",
       "│ Row │ x1 │ x2 │ x3 │\n",
       "├─────┼────┼────┼────┤\n",
       "│ 1   │ 1  │ 2  │ 3  │"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "DataFrame(transpose([1, 2, 3]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Pass a second argument to give the columns names."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>A</th><th>B</th><th>C</th></tr></thead><tbody><tr><th>1</th><td>1</td><td>4</td><td>7</td></tr><tr><th>2</th><td>2</td><td>5</td><td>8</td></tr><tr><th>3</th><td>3</td><td>6</td><td>9</td></tr></tbody></table>"
      ],
      "text/plain": [
       "3×3 DataFrames.DataFrame\n",
       "│ Row │ A │ B │ C │\n",
       "├─────┼───┼───┼───┤\n",
       "│ 1   │ 1 │ 4 │ 7 │\n",
       "│ 2   │ 2 │ 5 │ 8 │\n",
       "│ 3   │ 3 │ 6 │ 9 │"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "DataFrame([1:3, 4:6, 7:9], [:A, :B, :C])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here we create a `DataFrame` from a matrix,"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>x1</th><th>x2</th><th>x3</th><th>x4</th></tr></thead><tbody><tr><th>1</th><td>0.874557</td><td>0.777246</td><td>0.949467</td><td>0.697868</td></tr><tr><th>2</th><td>0.579164</td><td>0.816029</td><td>0.191466</td><td>0.0563065</td></tr><tr><th>3</th><td>0.280777</td><td>0.795716</td><td>0.201309</td><td>0.191633</td></tr></tbody></table>"
      ],
      "text/plain": [
       "3×4 DataFrames.DataFrame\n",
       "│ Row │ x1       │ x2       │ x3       │ x4        │\n",
       "├─────┼──────────┼──────────┼──────────┼───────────┤\n",
       "│ 1   │ 0.874557 │ 0.777246 │ 0.949467 │ 0.697868  │\n",
       "│ 2   │ 0.579164 │ 0.816029 │ 0.191466 │ 0.0563065 │\n",
       "│ 3   │ 0.280777 │ 0.795716 │ 0.201309 │ 0.191633  │"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "DataFrame(rand(3,4))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "and here we do the same but also pass column names."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>a</th><th>b</th><th>c</th><th>d</th></tr></thead><tbody><tr><th>1</th><td>0.627336</td><td>0.850384</td><td>0.225164</td><td>0.617465</td></tr><tr><th>2</th><td>0.645045</td><td>0.709581</td><td>0.0780468</td><td>0.0941601</td></tr><tr><th>3</th><td>0.48563</td><td>0.608378</td><td>0.777213</td><td>0.630866</td></tr></tbody></table>"
      ],
      "text/plain": [
       "3×4 DataFrames.DataFrame\n",
       "│ Row │ a        │ b        │ c         │ d         │\n",
       "├─────┼──────────┼──────────┼───────────┼───────────┤\n",
       "│ 1   │ 0.627336 │ 0.850384 │ 0.225164  │ 0.617465  │\n",
       "│ 2   │ 0.645045 │ 0.709581 │ 0.0780468 │ 0.0941601 │\n",
       "│ 3   │ 0.48563  │ 0.608378 │ 0.777213  │ 0.630866  │"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "DataFrame(rand(3,4), Symbol.('a':'d'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also construct an uninitialized DataFrame.\n",
    "\n",
    "Here we pass column types, names and number of rows; we get `missing` in column :C because `Any >: Missing`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>A</th><th>B</th><th>C</th></tr></thead><tbody><tr><th>1</th><td>147259888</td><td>1.08013e-319</td><td>missing</td></tr></tbody></table>"
      ],
      "text/plain": [
       "1×3 DataFrames.DataFrame\n",
       "│ Row │ A         │ B            │ C       │\n",
       "├─────┼───────────┼──────────────┼─────────┤\n",
       "│ 1   │ 147259888 │ 1.08013e-319 │ \u001b[90mmissing\u001b[39m │"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "DataFrame([Int, Float64, Any], [:A, :B, :C], 1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here we create a `DataFrame`, but column `:C` is #undef and Jupyter has problem with displaying it. (This works OK at the REPL.)\n",
    "\n",
    "This will be fixed in next release of DataFrames!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "ename": "UndefRefError",
     "evalue": "\u001b[91mUndefRefError: access to undefined reference\u001b[39m",
     "output_type": "error",
     "traceback": [
      "\u001b[91mUndefRefError: access to undefined reference\u001b[39m",
      "",
      "Stacktrace:",
      " [1] \u001b[1mgetindex\u001b[22m\u001b[22m\u001b[1m(\u001b[22m\u001b[22m::Array{String,1}, ::Int64\u001b[1m)\u001b[22m\u001b[22m at \u001b[1m.\\array.jl:554\u001b[22m\u001b[22m",
      " [2] \u001b[1mgetindex\u001b[22m\u001b[22m\u001b[1m(\u001b[22m\u001b[22m::DataFrames.DataFrame, ::Int64, ::Symbol\u001b[1m)\u001b[22m\u001b[22m at \u001b[1mD:\\Software\\JULIA_PKG\\v0.6\\DataFrames\\src\\dataframe\\dataframe.jl:275\u001b[22m\u001b[22m",
      " [3] \u001b[1mshow\u001b[22m\u001b[22m\u001b[1m(\u001b[22m\u001b[22m::IOContext{Base.AbstractIOBuffer{Array{UInt8,1}}}, ::MIME{Symbol(\"text/html\")}, ::DataFrames.DataFrame\u001b[1m)\u001b[22m\u001b[22m at \u001b[1mD:\\Software\\JULIA_PKG\\v0.6\\DataFrames\\src\\abstractdataframe\\io.jl:110\u001b[22m\u001b[22m",
      " [4] \u001b[1mlimitstringmime\u001b[22m\u001b[22m\u001b[1m(\u001b[22m\u001b[22m::MIME{Symbol(\"text/html\")}, ::DataFrames.DataFrame\u001b[1m)\u001b[22m\u001b[22m at \u001b[1mD:\\Software\\JULIA_PKG\\v0.6\\IJulia\\src\\inline.jl:24\u001b[22m\u001b[22m",
      " [5] \u001b[1mdisplay_dict\u001b[22m\u001b[22m\u001b[1m(\u001b[22m\u001b[22m::DataFrames.DataFrame\u001b[1m)\u001b[22m\u001b[22m at \u001b[1mD:\\Software\\JULIA_PKG\\v0.6\\IJulia\\src\\execute_request.jl:43\u001b[22m\u001b[22m",
      " [6] \u001b[1m(::Compat.#inner#17{Array{Any,1},IJulia.#display_dict,Tuple{DataFrames.DataFrame}})\u001b[22m\u001b[22m\u001b[1m(\u001b[22m\u001b[22m\u001b[1m)\u001b[22m\u001b[22m at \u001b[1mD:\\Software\\JULIA_PKG\\v0.6\\Compat\\src\\Compat.jl:385\u001b[22m\u001b[22m",
      " [7] \u001b[1mexecute_request\u001b[22m\u001b[22m\u001b[1m(\u001b[22m\u001b[22m::ZMQ.Socket, ::IJulia.Msg\u001b[1m)\u001b[22m\u001b[22m at \u001b[1mD:\\Software\\JULIA_PKG\\v0.6\\IJulia\\src\\execute_request.jl:186\u001b[22m\u001b[22m",
      " [8] \u001b[1m(::Compat.#inner#17{Array{Any,1},IJulia.#execute_request,Tuple{ZMQ.Socket,IJulia.Msg}})\u001b[22m\u001b[22m\u001b[1m(\u001b[22m\u001b[22m\u001b[1m)\u001b[22m\u001b[22m at \u001b[1mD:\\Software\\JULIA_PKG\\v0.6\\Compat\\src\\Compat.jl:385\u001b[22m\u001b[22m",
      " [9] \u001b[1meventloop\u001b[22m\u001b[22m\u001b[1m(\u001b[22m\u001b[22m::ZMQ.Socket\u001b[1m)\u001b[22m\u001b[22m at \u001b[1mD:\\Software\\JULIA_PKG\\v0.6\\IJulia\\src\\eventloop.jl:8\u001b[22m\u001b[22m",
      " [10] \u001b[1m(::IJulia.##14#17)\u001b[22m\u001b[22m\u001b[1m(\u001b[22m\u001b[22m\u001b[1m)\u001b[22m\u001b[22m at \u001b[1m.\\task.jl:335\u001b[22m\u001b[22m"
     ]
    }
   ],
   "source": [
    "DataFrame([Int, Float64, String], [:A, :B, :C], 1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To initialize a `DataFrame` with column names, but no rows use"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>A</th><th>B</th><th>C</th></tr></thead><tbody></tbody></table>"
      ],
      "text/plain": [
       "0×3 DataFrames.DataFrame\n"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "DataFrame([Int, Float64, String], [:A, :B, :C], 0) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This syntax gives us a quick way to create homogenous `DataFrame`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>x1</th><th>x2</th><th>x3</th><th>x4</th><th>x5</th></tr></thead><tbody><tr><th>1</th><td>202568848</td><td>199866128</td><td>1649267441664</td><td>0</td><td>439701264</td></tr><tr><th>2</th><td>147292368</td><td>147292368</td><td>0</td><td>0</td><td>148642768</td></tr><tr><th>3</th><td>147293072</td><td>147292432</td><td>0</td><td>0</td><td>439701424</td></tr></tbody></table>"
      ],
      "text/plain": [
       "3×5 DataFrames.DataFrame\n",
       "│ Row │ x1        │ x2        │ x3            │ x4 │ x5        │\n",
       "├─────┼───────────┼───────────┼───────────────┼────┼───────────┤\n",
       "│ 1   │ 202568848 │ 199866128 │ 1649267441664 │ 0  │ 439701264 │\n",
       "│ 2   │ 147292368 │ 147292368 │ 0             │ 0  │ 148642768 │\n",
       "│ 3   │ 147293072 │ 147292432 │ 0             │ 0  │ 439701424 │"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "DataFrame(Int, 3, 5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This example is similar, but has nonhomogenous columns."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>x1</th><th>x2</th></tr></thead><tbody><tr><th>1</th><td>148644688</td><td>2.17246e-315</td></tr><tr><th>2</th><td>147261328</td><td>2.17242e-315</td></tr><tr><th>3</th><td>147261328</td><td>2.17246e-315</td></tr><tr><th>4</th><td>0</td><td>2.20078e-315</td></tr></tbody></table>"
      ],
      "text/plain": [
       "4×2 DataFrames.DataFrame\n",
       "│ Row │ x1        │ x2           │\n",
       "├─────┼───────────┼──────────────┤\n",
       "│ 1   │ 148644688 │ 2.17246e-315 │\n",
       "│ 2   │ 147261328 │ 2.17242e-315 │\n",
       "│ 3   │ 147261328 │ 2.17246e-315 │\n",
       "│ 4   │ 0         │ 2.20078e-315 │"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "DataFrame([Int, Float64], 4)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, we can create a `DataFrame` by copying an existing `DataFrame`.\n",
    "\n",
    "Note that `copy` creates a shallow copy."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(false, false, true)"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y = DataFrame(x)\n",
    "z = copy(x)\n",
    "(x === y), (x === z), isequal(x, z)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Conversion to a matrix\n",
    "\n",
    "Let's start by creating a `DataFrame` with two rows and two columns."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>x</th><th>y</th></tr></thead><tbody><tr><th>1</th><td>1</td><td>A</td></tr><tr><th>2</th><td>2</td><td>B</td></tr></tbody></table>"
      ],
      "text/plain": [
       "2×2 DataFrames.DataFrame\n",
       "│ Row │ x │ y │\n",
       "├─────┼───┼───┤\n",
       "│ 1   │ 1 │ A │\n",
       "│ 2   │ 2 │ B │"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = DataFrame(x=1:2, y=[\"A\", \"B\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can create a matrix by passing this `DataFrame` to `Matrix`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2×2 Array{Any,2}:\n",
       " 1  \"A\"\n",
       " 2  \"B\""
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Matrix(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This would work even if the `DataFrame` had some `missing`s:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>x</th><th>y</th></tr></thead><tbody><tr><th>1</th><td>1</td><td>missing</td></tr><tr><th>2</th><td>2</td><td>B</td></tr></tbody></table>"
      ],
      "text/plain": [
       "2×2 DataFrames.DataFrame\n",
       "│ Row │ x │ y       │\n",
       "├─────┼───┼─────────┤\n",
       "│ 1   │ 1 │ \u001b[90mmissing\u001b[39m │\n",
       "│ 2   │ 2 │ B       │"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = DataFrame(x=1:2, y=[missing,\"B\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2×2 Array{Any,2}:\n",
       " 1  missing\n",
       " 2  \"B\"    "
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Matrix(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the two previous matrix examples, Julia created matrices with elements of type `Any`. We can see more clearly that the type of matrix is inferred when we pass, for example, a `DataFrame` of integers to `Matrix`, creating a 2D `Array` of `Int64`s:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>x</th><th>y</th></tr></thead><tbody><tr><th>1</th><td>1</td><td>3</td></tr><tr><th>2</th><td>2</td><td>4</td></tr></tbody></table>"
      ],
      "text/plain": [
       "2×2 DataFrames.DataFrame\n",
       "│ Row │ x │ y │\n",
       "├─────┼───┼───┤\n",
       "│ 1   │ 1 │ 3 │\n",
       "│ 2   │ 2 │ 4 │"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = DataFrame(x=1:2, y=3:4)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2×2 Array{Int64,2}:\n",
       " 1  3\n",
       " 2  4"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Matrix(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this next example, Julia correctly identifies that `Union` is needed to express the type of the resulting `Matrix` (which contains `missing`s)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>x</th><th>y</th></tr></thead><tbody><tr><th>1</th><td>1</td><td>missing</td></tr><tr><th>2</th><td>2</td><td>4</td></tr></tbody></table>"
      ],
      "text/plain": [
       "2×2 DataFrames.DataFrame\n",
       "│ Row │ x │ y       │\n",
       "├─────┼───┼─────────┤\n",
       "│ 1   │ 1 │ \u001b[90mmissing\u001b[39m │\n",
       "│ 2   │ 2 │ 4       │"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = DataFrame(x=1:2, y=[missing,4])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2×2 Array{Union{Int64, Missings.Missing},2}:\n",
       " 1   missing\n",
       " 2  4       "
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Matrix(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that we can't force a conversion of `missing` values to `Int`s!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "ename": "LoadError",
     "evalue": "\u001b[91mcannot convert a DataFrame containing missing values to array (found for column y)\u001b[39m",
     "output_type": "error",
     "traceback": [
      "\u001b[91mcannot convert a DataFrame containing missing values to array (found for column y)\u001b[39m",
      "",
      "Stacktrace:",
      " [1] \u001b[1mconvert\u001b[22m\u001b[22m\u001b[1m(\u001b[22m\u001b[22m::Type{Array{Int64,2}}, ::DataFrames.DataFrame\u001b[1m)\u001b[22m\u001b[22m at \u001b[1mD:\\Software\\JULIA_PKG\\v0.6\\DataFrames\\src\\abstractdataframe\\abstractdataframe.jl:626\u001b[22m\u001b[22m",
      " [2] \u001b[1mArray{Int64,2}\u001b[22m\u001b[22m\u001b[1m(\u001b[22m\u001b[22m::DataFrames.DataFrame\u001b[1m)\u001b[22m\u001b[22m at \u001b[1m.\\sysimg.jl:77\u001b[22m\u001b[22m"
     ]
    }
   ],
   "source": [
    "Matrix{Int}(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Handling of duplicate column names\n",
    "\n",
    "We can pass the `makeunique` keyword argument to allow passing duplicate names (they get deduplicated)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>a</th><th>a_2</th><th>a_1</th></tr></thead><tbody><tr><th>1</th><td>1</td><td>2</td><td>3</td></tr></tbody></table>"
      ],
      "text/plain": [
       "1×3 DataFrames.DataFrame\n",
       "│ Row │ a │ a_2 │ a_1 │\n",
       "├─────┼───┼─────┼─────┤\n",
       "│ 1   │ 1 │ 2   │ 3   │"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = DataFrame(:a=>1, :a=>2, :a_1=>3; makeunique=true)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Otherwise, duplicates will not be allowed in the future."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[1m\u001b[33mWARNING: \u001b[39m\u001b[22m\u001b[33mDuplicate variable names are deprecated: pass makeunique=true to add a suffix automatically.\u001b[39m\n",
      "Stacktrace:\n",
      " [1] \u001b[1mdepwarn\u001b[22m\u001b[22m\u001b[1m(\u001b[22m\u001b[22m::String, ::Symbol\u001b[1m)\u001b[22m\u001b[22m at \u001b[1m.\\deprecated.jl:70\u001b[22m\u001b[22m\n",
      " [2] \u001b[1m#make_unique#3\u001b[22m\u001b[22m\u001b[1m(\u001b[22m\u001b[22m::Bool, ::Function, ::Array{Symbol,1}\u001b[1m)\u001b[22m\u001b[22m at \u001b[1mD:\\Software\\JULIA_PKG\\v0.6\\DataFrames\\src\\other\\utils.jl:61\u001b[22m\u001b[22m\n",
      " [3] \u001b[1m(::DataFrames.#kw##make_unique)\u001b[22m\u001b[22m\u001b[1m(\u001b[22m\u001b[22m::Array{Any,1}, ::DataFrames.#make_unique, ::Array{Symbol,1}\u001b[1m)\u001b[22m\u001b[22m at \u001b[1m.\\<missing>:0\u001b[22m\u001b[22m\n",
      " [4] \u001b[1m#Index#6\u001b[22m\u001b[22m at \u001b[1mD:\\Software\\JULIA_PKG\\v0.6\\DataFrames\\src\\other\\index.jl:12\u001b[22m\u001b[22m [inlined]\n",
      " [5] \u001b[1m(::Core.#kw#Type)\u001b[22m\u001b[22m\u001b[1m(\u001b[22m\u001b[22m::Array{Any,1}, ::Type{DataFrames.Index}, ::Array{Symbol,1}\u001b[1m)\u001b[22m\u001b[22m at \u001b[1m.\\<missing>:0\u001b[22m\u001b[22m\n",
      " [6] \u001b[1m#DataFrame#47\u001b[22m\u001b[22m\u001b[1m(\u001b[22m\u001b[22m::Bool, ::Type{T} where T, ::Pair{Symbol,Int64}, ::Vararg{Pair{Symbol,Int64},N} where N\u001b[1m)\u001b[22m\u001b[22m at \u001b[1mD:\\Software\\JULIA_PKG\\v0.6\\DataFrames\\src\\dataframe\\dataframe.jl:126\u001b[22m\u001b[22m\n",
      " [7] \u001b[1mDataFrames.DataFrame\u001b[22m\u001b[22m\u001b[1m(\u001b[22m\u001b[22m::Pair{Symbol,Int64}, ::Pair{Symbol,Int64}, ::Pair{Symbol,Int64}\u001b[1m)\u001b[22m\u001b[22m at \u001b[1mD:\\Software\\JULIA_PKG\\v0.6\\DataFrames\\src\\dataframe\\dataframe.jl:124\u001b[22m\u001b[22m\n",
      " [8] \u001b[1minclude_string\u001b[22m\u001b[22m\u001b[1m(\u001b[22m\u001b[22m::String, ::String\u001b[1m)\u001b[22m\u001b[22m at \u001b[1m.\\loading.jl:522\u001b[22m\u001b[22m\n",
      " [9] \u001b[1mexecute_request\u001b[22m\u001b[22m\u001b[1m(\u001b[22m\u001b[22m::ZMQ.Socket, ::IJulia.Msg\u001b[1m)\u001b[22m\u001b[22m at \u001b[1mD:\\Software\\JULIA_PKG\\v0.6\\IJulia\\src\\execute_request.jl:158\u001b[22m\u001b[22m\n",
      " [10] \u001b[1m(::Compat.#inner#17{Array{Any,1},IJulia.#execute_request,Tuple{ZMQ.Socket,IJulia.Msg}})\u001b[22m\u001b[22m\u001b[1m(\u001b[22m\u001b[22m\u001b[1m)\u001b[22m\u001b[22m at \u001b[1mD:\\Software\\JULIA_PKG\\v0.6\\Compat\\src\\Compat.jl:385\u001b[22m\u001b[22m\n",
      " [11] \u001b[1meventloop\u001b[22m\u001b[22m\u001b[1m(\u001b[22m\u001b[22m::ZMQ.Socket\u001b[1m)\u001b[22m\u001b[22m at \u001b[1mD:\\Software\\JULIA_PKG\\v0.6\\IJulia\\src\\eventloop.jl:8\u001b[22m\u001b[22m\n",
      " [12] \u001b[1m(::IJulia.##14#17)\u001b[22m\u001b[22m\u001b[1m(\u001b[22m\u001b[22m\u001b[1m)\u001b[22m\u001b[22m at \u001b[1m.\\task.jl:335\u001b[22m\u001b[22m\n",
      "while loading In[28], in expression starting on line 1\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>a</th><th>a_2</th><th>a_1</th></tr></thead><tbody><tr><th>1</th><td>1</td><td>2</td><td>3</td></tr></tbody></table>"
      ],
      "text/plain": [
       "1×3 DataFrames.DataFrame\n",
       "│ Row │ a │ a_2 │ a_1 │\n",
       "├─────┼───┼─────┼─────┤\n",
       "│ 1   │ 1 │ 2   │ 3   │"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = DataFrame(:a=>1, :a=>2, :a_1=>3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A constructor that is passed column names as keyword arguments is a corner case.\n",
    "You cannot pass `makeunique` to allow duplicates here."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[1m\u001b[33mWARNING: \u001b[39m\u001b[22m\u001b[33mDuplicate variable names are deprecated: pass makeunique=true to add a suffix automatically.\u001b[39m\n",
      "Stacktrace:\n",
      " [1] \u001b[1mdepwarn\u001b[22m\u001b[22m\u001b[1m(\u001b[22m\u001b[22m::String, ::Symbol\u001b[1m)\u001b[22m\u001b[22m at \u001b[1m.\\deprecated.jl:70\u001b[22m\u001b[22m\n",
      " [2] \u001b[1m#make_unique#3\u001b[22m\u001b[22m\u001b[1m(\u001b[22m\u001b[22m::Bool, ::Function, ::Array{Symbol,1}\u001b[1m)\u001b[22m\u001b[22m at \u001b[1mD:\\Software\\JULIA_PKG\\v0.6\\DataFrames\\src\\other\\utils.jl:61\u001b[22m\u001b[22m\n",
      " [3] \u001b[1m(::DataFrames.#kw##make_unique)\u001b[22m\u001b[22m\u001b[1m(\u001b[22m\u001b[22m::Array{Any,1}, ::DataFrames.#make_unique, ::Array{Symbol,1}\u001b[1m)\u001b[22m\u001b[22m at \u001b[1m.\\<missing>:0\u001b[22m\u001b[22m\n",
      " [4] \u001b[1m#Index#6\u001b[22m\u001b[22m at \u001b[1mD:\\Software\\JULIA_PKG\\v0.6\\DataFrames\\src\\other\\index.jl:12\u001b[22m\u001b[22m [inlined]\n",
      " [5] \u001b[1m(::Core.#kw#Type)\u001b[22m\u001b[22m\u001b[1m(\u001b[22m\u001b[22m::Array{Any,1}, ::Type{DataFrames.Index}, ::Array{Symbol,1}\u001b[1m)\u001b[22m\u001b[22m at \u001b[1m.\\<missing>:0\u001b[22m\u001b[22m\n",
      " [6] \u001b[1m#DataFrame#47\u001b[22m\u001b[22m\u001b[1m(\u001b[22m\u001b[22m::Bool, ::Type{T} where T, ::Pair{Symbol,Int64}, ::Vararg{Pair{Symbol,#s8} where #s8,N} where N\u001b[1m)\u001b[22m\u001b[22m at \u001b[1mD:\\Software\\JULIA_PKG\\v0.6\\DataFrames\\src\\dataframe\\dataframe.jl:126\u001b[22m\u001b[22m\n",
      " [7] \u001b[1mDataFrames.DataFrame\u001b[22m\u001b[22m\u001b[1m(\u001b[22m\u001b[22m::Pair{Symbol,Int64}, ::Pair{Symbol,Int64}, ::Pair{Symbol,Bool}\u001b[1m)\u001b[22m\u001b[22m at \u001b[1mD:\\Software\\JULIA_PKG\\v0.6\\DataFrames\\src\\dataframe\\dataframe.jl:124\u001b[22m\u001b[22m\n",
      " [8] \u001b[1m#DataFrame#56\u001b[22m\u001b[22m at \u001b[1mD:\\Software\\JULIA_PKG\\v0.6\\DataFrames\\src\\dataframe\\dataframe.jl:145\u001b[22m\u001b[22m [inlined]\n",
      " [9] \u001b[1m(::Core.#kw#Type)\u001b[22m\u001b[22m\u001b[1m(\u001b[22m\u001b[22m::Array{Any,1}, ::Type{DataFrames.DataFrame}\u001b[1m)\u001b[22m\u001b[22m at \u001b[1m.\\<missing>:0\u001b[22m\u001b[22m\n",
      " [10] \u001b[1minclude_string\u001b[22m\u001b[22m\u001b[1m(\u001b[22m\u001b[22m::String, ::String\u001b[1m)\u001b[22m\u001b[22m at \u001b[1m.\\loading.jl:522\u001b[22m\u001b[22m\n",
      " [11] \u001b[1mexecute_request\u001b[22m\u001b[22m\u001b[1m(\u001b[22m\u001b[22m::ZMQ.Socket, ::IJulia.Msg\u001b[1m)\u001b[22m\u001b[22m at \u001b[1mD:\\Software\\JULIA_PKG\\v0.6\\IJulia\\src\\execute_request.jl:158\u001b[22m\u001b[22m\n",
      " [12] \u001b[1m(::Compat.#inner#17{Array{Any,1},IJulia.#execute_request,Tuple{ZMQ.Socket,IJulia.Msg}})\u001b[22m\u001b[22m\u001b[1m(\u001b[22m\u001b[22m\u001b[1m)\u001b[22m\u001b[22m at \u001b[1mD:\\Software\\JULIA_PKG\\v0.6\\Compat\\src\\Compat.jl:385\u001b[22m\u001b[22m\n",
      " [13] \u001b[1meventloop\u001b[22m\u001b[22m\u001b[1m(\u001b[22m\u001b[22m::ZMQ.Socket\u001b[1m)\u001b[22m\u001b[22m at \u001b[1mD:\\Software\\JULIA_PKG\\v0.6\\IJulia\\src\\eventloop.jl:8\u001b[22m\u001b[22m\n",
      " [14] \u001b[1m(::IJulia.##14#17)\u001b[22m\u001b[22m\u001b[1m(\u001b[22m\u001b[22m\u001b[1m)\u001b[22m\u001b[22m at \u001b[1m.\\task.jl:335\u001b[22m\u001b[22m\n",
      "while loading In[29], in expression starting on line 1\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>a</th><th>a_1</th><th>makeunique</th></tr></thead><tbody><tr><th>1</th><td>1</td><td>2</td><td>true</td></tr></tbody></table>"
      ],
      "text/plain": [
       "1×3 DataFrames.DataFrame\n",
       "│ Row │ a │ a_1 │ makeunique │\n",
       "├─────┼───┼─────┼────────────┤\n",
       "│ 1   │ 1 │ 2   │ true       │"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = DataFrame(a=1, a=2, makeunique=true)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Julia 0.6.0",
   "language": "julia",
   "name": "julia-0.6"
  },
  "language_info": {
   "file_extension": ".jl",
   "mimetype": "application/julia",
   "name": "julia",
   "version": "0.6.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}
